Medical hyperspectral imaging for evaluation of tissue and tumor

ABSTRACT

Apparatus and methods for hyperspectral imaging analysis that assists in real and near-real time assessment of biological tissue condition, viability, and type, and monitoring the above overtime. Embodiments of the invention are particularly useful in surgery, clinical procedures, tissue assessment, diagnostic procedures, health monitoring, and medical evaluations, especially in the detection and treatment of cancer.

REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/678,454 entitled Medical Hyperspectral Imaging forEvaluation of Tissue and Tumor, filed Nov. 15, 2012, which is acontinuation of U.S. patent application Ser. No. 11/288,410 entitledMedical Hyperspectral Imaging for Evaluation of Tissue and Tumor, filedNov. 29, 2005, now U.S. Pat. No. 8,320,996, which claims priority toU.S. Provisional Patent Application Ser. No. 60/631,135 entitledHyperspectral Imaging in Medical Applications, filed Nov. 29, 2004, Ser.No. 60/667,678 entitled Hyperspectral Imaging in Breast Cancer, filed onApr. 4, 2005, and Ser. No. 60/732,146 entitled Hyperspectral Analysisfor the Detection of Lymphoma, filed on Nov. 2, 2005, which are herebyincorporated by reference.

FIELD OF THE INVENTION

The invention is directed to a hyperspectral imaging analysis thatassists in real and near-real time assessment of biological tissuecondition, viability, and type, and monitoring the above over time.Embodiments of the invention are particularly useful in surgery,clinical procedures, tissue assessment, diagnostic procedures, healthmonitoring, and medical evaluations.

BACKGROUND OF THE INVENTION

In 2005, 212,000 new cases of breast cancer are expected, andapproximately 40,000 women will die of the disease.¹ Recent nationalfigures indicate that approximately 45% of patients with breast, cancerundergo primary surgical treatment with mastectomy.² The use of breastconserving treatment (lumpectomy and radiation therapy; BCT) isincreasing as primary surgical treatment for breast cancer as long termstudies have documented the efficacy of BCT.³ BCT is often followed bysystemic therapy with chemotherapy, hormone therapy, or both. Aprerequisite for BCT is complete removal of the cancer, documented bynegative margins on pathologic evaluation of fee lumpectomy specimen.The presence of positive margins is associated with increased localrecurrence (LR) rates, 10-15% vs. 1-10% with negative margins.⁴ Acompeting interest is the preservation of breast tissue to minimizedeformity.

The importance of local recurrence is controversial. Early studiessuggested that LR does hot translate into death from disease.⁴ However,recent data showing lower LR rates and survival benefit by addingradiation therapy to mastectomy for patients with higher stage cancersindicate the potential importance of freedom from local recurrence.⁵ Inaddition, LR contributes to significant local, morbidity, usuallyrequiring a mastectomy. Finally LR contributes to the cost of care andanxiety for the patient. Despite these issues. 20-60% of patientsundergoing BCT are found to have positive margins requiring additionalsurgical procedures, either re-excisional lumpectomies, or mastectomy.⁶These additional procedures result in increased cost, increased anxietyfor the patient, and, importantly, a delay in initiation of importantsystemic chemotherapy or radiation therapy.

Although many patients undergoing excisional breast biopsy are found tonot have cancer, the wider use of pre-operative core needle biopsy hasincreased the preoperative diagnosis of invasive breast cancer (SBC) orductal carcinoma in situ (DOS). At operation, the surgeon attempts tocompletely resect the cancer with negative microscopic margins, butfeces several difficulties, DOS often is associated with grossly normalappearing breast tissue and no mass. Breast cancers presenting as a massallow the surgeon to feel and see the area to be excised. However, themicroscopic extent of disease is difficult to gauge. Frozen sectionanalysis of breast biopsy margins is difficult and unreliable, becausethe fat content of breast tissue results in difficulty in sectioningfrozen specimens. Even after standard tissue preparation over 2 days,one estimate is that more than 1,000 slices of a 2 cm biopsy specimenwould be necessary to ensure completely negative, margins. Pathologistshave attempted to peel the external surface of a permanently fixedspecimen, as one might peel an orange, to evaluate the entirety of thespecimen margin. This is difficult, and impractical in mostinstitutions, but also does not provide real time information while thepatient is in the operating room.

For all of these reasons, surgeons have adopted several techniques toincrease the likelihood of negative margins. They may ink the entirespecimen with a single colored ink in the operating room or in thepathology suite with the pathologist. More recently, multiple coloredinks have been used to mark the six sides of a cuboid breast specimen.The former method does not allow re-resection of a specific positivemargin and results in resection of a larger volume of breast tissuesince the cavity side with a positive margin is not known. With bothapproaches, the ink may creep into crevices, resulting in falselypositive margins. With fee multi-colored approach, inks may runtogether, resulting in confusion as to the location of a specificpositive margin.

Many surgeons perform wide excisions, potentially resulting insignificant breast deformity that is added, to by the breast, shrinkageassociated with radiation therapy. An effective and widely used methodto enhance the likelihood of negative margins requires the surgeon,after excision of the tumor bearing specimen, to take additional slicesof breast tissue from the four sides aid deep surface of the open breastcavity, and submit these additional “margins” separately as the finalmargins. This approach eliminates any confusion as to the location ofthe margin. When this technique is used, additional cancer is found in20% of additional margins when the margins of the original specimen werenegative.⁵ Regardless of the technique, final pathology evaluation maytake up to one week. This delay results in patient anxiety and longertime to completion of the patient's surgical treatment. A method forreliable, infra-operative margin evaluation would be of great value forbreast cancer surgery.

Sentinel lymph node biopsy (SLNB) has replaced elective lymph nodedissection (ELND) of the ipsilateral axilla for patients with invasivebreast cancer. Because of fee high negative predictive value of SLNBpatients with negative sentinel nodes are spared the need for a completeaxillary dissection, with its attendant morbidity and cost. Patientswith positive nodes may undergo complete axillary dissectionsynchronously if a frozen section pathology report is positive. Theaccuracy of sentinel node evaluation by frozen section is problematic,⁶with a significant false negative risk, when compared to the finalreport. To avoid giving patients bad news after an initial favorablereport, many surgeons avoid frozen section entirely, waiting up to aweek for the final pathology evaluation to decide whether a patientneeds additional surgery. That additional surgery may take place 1-2weeks later. Lymph, nodes containing malignant, cells may have alteredblood flow, which may fee seen by Hyperspectral imaging. A reliable,real time method which accurately predicts lymph node metastasis wouldallow synchronous and complete management of the axilla, and reduce oreliminate additional anesthesias and operations.⁷

Lymphomas, which include Hodgkin's disease and noo-Modgkin's lymphoma,are the fifth most common type of cancer diagnosed and the sixth mostcommon cause of cancer death in the United States. Of the two basiclymphoma types, non-Hodgkin's lymphoma is the more common, with 16,000new cases diagnosed annually.⁸ The age-adjusted incidence rate ofnon-Hodgkin's lymphoma among non-Hispanic white men (the demographicgroup with the greatest preponderance) is 19.1 per 100*000 and amongnon-Hispanic white women are 12.0 per 100,000. Not unexpectedly,incidence rates increase with age, with a 5-fold increase from ages30-54 to 70 and older for non-Hispanic white men, but 16-fold amongFilipino women, the group with the greatest increase. However, leukemiaand lymphoma also account for about half of the new cancer cases inchildren. Preclinical detection and intervention are likely to achieve areduction in these rates. Patients already treated for lymphoma are atthe greatest risk. Significantly, a study of patients monitoredintensively for relapse (by physical examination, serum, analysis, chestX-ray, gallium and CT scanning, ultrasound and bone marrow biopsy)determined feat, in 91% of patients, relapse was detected at unscheduledvisits for symptomatic disease.⁹ Furthermore, standard chemotherapy iseffective in only 40% of patients. Clearly, new and more effectivemeasures are needed, such as high resolution hyperspectral imaging ofphysiologic biomarkers for early detection of relapse.

A method, for non-invasive evaluation of the progression ofnon-Hodgkin's lymphoma (NHL) and responses to therapy would be highlyadvantageous, having utility as both a non-destructive animal research,tool, and as a non-invasive clinical tool, which greatly improvediagnostic efficiency. Disease progression can be evaluated in solidtissue such as the spleen and from monitoring leukemic cells in bloodand lymph nodes. In addition to monitoring systemic microvasculareffects induced by the disease.

Differentiating between types of tissue is useful in the medical andsurgical arenas. This includes differentiating between types of normaltissue or between varieties of normal tissue types and distinguishingthem from tumor tissue.

SUMMARY OF INVENTION

The present invention overcomes the problems and disadvantagesassociated with current strategies and designs and provides new toolsand methods for defecting and assessing cancer in human tissue.

One embodiment of the invention is directed to a medical instrumentcomprising a first-stage optic responsive to illumination of a tissue, aspectral separator, one or more polarizers, an imaging sensor, adiagnostic processor, a filter control, interface, and a general-purposeoperating module. Preferably, the spectral separator is opticallyresponsive to the first-stage optic and has a control input, thepolarizer compiles a plurality of light beams into a plane ofpolarization before entering the imaging sensor, the imaging sensor isoptically responsive to the spectral separator and has an image dataoutput, the diagnostic processor comprises an image acquisitioninterface with an Input responsive to the imaging sensor and one or morediagnostic protocol modules wherein each diagnostic protocol modulecontains a set of instructions for operating the spectral separator andfor operating the filter control interface, the filter control interfacecomprises a control output provided to the control input of the spectralseparator, which directs the spectral separator independently of theillumination to receive one or more wavelengths of the illumination toprovide multispectral or hyperspectral information as determined by theset of instructions provided by the one or more diagnostic protocolmodule, and the general-purpose operating module performs filtering andacquiring steps one or more times depending on the set of instructionsprovided by the one or more diagnostic protocol modules.

The instrument may also comprise a second-stage optic responsive toIllumination of the tissue. Preferably, the one or more wavelengths ofillumination is one or a combination of UV, visible, NIR, and IR. Inpreferred embodiments, the multispectral or hyperspectral informationdetermines one or more of presence of cancer for screening or diagnosis,presence of residual cancer in a surgical excision bed, and cancerprogression, preferably wherein the presence of cancer is breast,lymphoma or any cancer readily visualized by hypes-spectral imaging.Preferred embodiments include the multispectral or hyperspectralinformation applied laparoscopically, thoracoscopically,cystoscopically, hysteroscopically, bronchoscopically, ormediastinoscopically to assess presence of tumor, adequacy of surgicalresection or nodal or intracavitary spread.

The cancer progression detected may be one of tumor stage grading andmicrovascular changes in any vascular tissue such as skin, eye, ear,nodularity. The presence of cancer detected may fee one of presence oftumor, presence of residual tumor at margin of resection, lymph nodeassessment, primary diagnosis, and tumor grade or invasiveness.

Another embodiment is directed to the set of instructions comprisingpreprocessing the hyperspectral information, building a visual image,defining a region of interest of the tissue, converting allhyperspectral image intensities into units of optical density by takinga negative logarithm of each decimal base, decomposing a spectra foreach pixel into several independent components, determining three planesfor an RGB pseudo-color image, determining a sharpness factor plane,converting the RGB pseudo-color image to ahue-saturation-value/intensity (HSV/I) image having a plane, scaling thehue-saturation-value/intensity image plane with the sharpness factorplane, converting the hue-saturation-value/intensity image hack to theRGB pseudo-color image, removing outliers beyond a standard deviationand stretching image between 0 and 1 displaying the region of interestin pseudo-colors; and characterizing a metabolic state of fee tissue ofinterest.

The region of interest may be a region or an entire field of view.Preferably, determining the three planes for an RGB pseudo-color imagecomprises one or more characteristic features of the spectra.Preferably, determining a sharpness factor plane comprises a combinationof the images at different wavelengths, preferably by taking a ratio ofa yellow plane in the range of about 550-580 nm to a green plane in therange of about 495-525 nm, or by taking a combination of oxyhemoglobinand deoxyhemoglobin spectral components, or by taking a ratio between awavelength in the red region in the range 615-710 nm and a wavelength inthe yellow region in the range of about 550-580 nm or in the orangeregion in the range of about 580-815 nm. Preferably, outliers areremoved beyond a standard deviation, preferably three standarddeviations. The region of interest is displayed in pseudo-colors,performed with one of in combination with a color photo image of asubject, or in addition to a color photo image of a subject or byprojecting the pseudo-color image onto the observed surface.

Another embodiment of the invention is directed to a method fordetecting cancer in tissue comprising preprocessing the hyperspectralinformation, building a visual image, defining a region of interest ofthe tissue, converting all hyperspectral image intensities into units ofoptical density by taking a negative logarithm, of each, decimal base,decomposing a spectra for each pixel into several independentcomponents, determining three planes for an RGB pseudo-color image,determining a sharpness factor plane, converting the RGB pseudo-colorimage to a hue-saturation-value/intensity (HSV/I) image having a plane,scaling the hue-saturation-value/intensity image plane with thesharpness factor plane, converting the hue-saturation-value/intensityimage hack to the RGB pseudo-color image, removing outliers beyond astandard deviation and stretching image between 0 and 1, displaying theregion of interest in pseudo-colors, and characterizing a metabolicstate of the tissue of interest.

The region, of interest may be a region or an entire field of view.Preferably, determining the three planes for an RGB pseudo-color imagecomprises one or more characteristic features of die spectra.Preferably, determining a sharpness factor plane comprises a combinationof the images at different wavelengths, preferably by taking a ratio ofa yellow plane in the range of about 550-580 nm to a green plane in therange of about 495-525 nm, or by taking a combination of oxyhemoglobinand deoxyhemoglobin spectral components, or by taking a ratio between awavelength in the red region in the range 615-710 nm and a wavelength,in the yellow region in the range of about 550-580 nm or in the orangeregion in the range of about 580-615 nm. Preferably, outliers areremoved beyond a standard deviation, preferably three standarddeviations. The region of interest is displayed in pseudo-colors,performed with one of in combination with a color photo image of asubject, or in addition to a color photo image of a subject, or byprojecting the pseudo-color image onto the observed surface.

Another embodiment is directed to a medical instrument comprising animage projector, one or more remote lights, a remote control device anda real-time data processing package.

Other embodiments and advantages of the invention are set forth in partin the description, which follows, and in part, may be obvious from,this description, or may be learned from the practice of the invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1: Block diagram depicting a portable hyperspectral imagingapparatus.

FIG. 2: Absorption spectra in the visible light wavelength range fromdifferent tissue types as recorded, by a single pixel on an imagerecording devise.

FIG. 3: A sequence of images comparing color pictures of the operationsite, where (A) is the field of view as seen by surgeon, (B) is a slightmagnification to show residual tumors intentionally left in the tamerbed, and (C) is the hyperspectral solution image. MHSI identifiesdifferent tissue types for the surgeon by displaying results of thealgorithm using pseudo-color images (C) that highlight and amplifyvisibility of tumor tissue.

FIG. 4: Forty (40) micron resolution was available via real time digitalzoom immediately after image acquisition from, a stationary MHSI deviceplaced over the surgical field (see FIG. 1, 4 cm×6 cm field of view),enabling the operating surgeon to review images on the MHSI computerscreen during the procedure to obtain an indication of tissue type(tumor vs. normal) and to evaluate surrounding microvasculature.

FIGS. 5A, 5B, 5C and 5D respectively show FIG. 5A: MHSI Images ofundisturbed tumors, exposed tumors, small residual tumors intentionallyleft in bed for easy interpretations, and tumor beds after completeresection.

FIG. 6: MHSI distinguishes hematoma from tumor. Hematoma andextravasated blood (red/pink/orange in left panel) are often visuallyindistinguishable from residual tumor to the eye of the surgeon or in asimple color picture, whereas in the HyperMed MHSI pseudo-color image(right panel) blood is seen as black, oxygenated tissue as pink, andresidual tumor as cyan-blue masses.

FIG. 7: MHSI images of tissue representative of each group graded frontnormal (grade 0) to carcinoma with invasion (grade 4). Scale for eachimage is 13-by-13 mm. Normal tissue (grade 0) is shown on left panel.The benign tumor and intraductal carcinomas (grades 1 and 2) havesimilar masses; typically benign tumors (I) have smaller size than theintraductal carcinomas (middle panel). More advanced carcinomas:papillary and cribriform (grade 3) and carcinoma with invasion (grade 4)are represented by masses of solid color where internal structure of thetumor seems dense and does not show the details (last two panels).

FIG. 8: MHSI highly correlated with histology, MHSI image from tumor insitu (4×3 cm) was collected by surgeon. Resected tumor and surroundingtissue (5×7 mm) was evaluated by histopathology after resection.Microscopic images with further resolution are displayed. Note thathistologic features are mirrored in MHSI image.

FIG. 9: Standard visible images of side of control mouse, upper paneland mouse with late stage lymphoma (8 days post transplant), lowerpanel. Note small darker region representing spleen in control mouse andmuch larger spleen in lymphomatous mouse. Mice had undergone localchemical hair removal 24 hours prior but a residual patch of hair inlymphomatous mouse remains which obscures MHSI measurement.

FIG. 10: Spectra representing marked differences in all tissues examinedin lymphomatous (solid) and control (dotted) mice.

FIG. 11: MHSI images reporting oxyhemoglobin, deoxyhemoglobin, totalhemoglobin and tissue oxygen saturation. Control mouse is above andlymphomatous mouse is below. Note marked decrease in oxyhemoglobin,deoxyhemoglobin, and total hemoglobin in skin of lymphomatous mouse andsimilar although lesser changes in the spleen. Note also nodular regionin spleen which may represent a lymphomatous nodule.

FIG. 12: Standard visible images of head of control mouse (left panel)and mouse with late stage lymphoma (right panel; 8 dayspost-transplant).

FIG. 13: Spectra representing marked differences in all tissues examinedin lymphomatous (solid) and control (dotted) mice.

FIG. 14: Colorized MHSI images reporting oxyhemoglobin, deoxyhemoglobintotal hemoglobin and tissue oxygen saturation of head of mice fromvisible images above. Only relevant areas are ears and eyes, becauseremainder of head is covered by hair. Note marked decrease inoxyhemoglobin and oxygen saturation in eye, ear skin and blood vesselsin lymphomatous mouse. There are similar but much less markeddifferences in deoxy and total hemoglobin images. These changes areconsistent with both a decrease in red cell volume and in a decrease inflow with greater oxygen extraction by the tissue. A tumor signature forlymphoma is able to be defined within the ear vessels.

DESCRIPTION OF THE INVENTION Hyperspectral Imaging System

Hyperspectral imaging (HSI) is a novel, method of “imaging spectroscopy”that generates a map of a region of interest based on local chemicalcomposition. HSI has been used in non-medical applications includingsatellite investigation to indicate areas of chemical weapons productionand to assess the condition of agricultural fields, HSI has recentlybeen applied to the investigation of physiologic and pathologic changesin living tissue in animal and human studies to provide information asto the health or disease of tissue that is otherwise unavailable. MHSIhas been shown to accurately predict viability and survival of tissuedeprived of adequate perfusion, and to differentiate diseased tissue(e.g. tumor) and growth due to cancerous angiogenesis in a rat modelsystem of breast cancer.

HSI is a remote sensing technology in which a 2-dimensional image iscreated having spectral data inherent in each pixel. These stacks ofimages comprise what is called a hypercube. It is possible to correlatethe spectrum of each pixel with the presence and concentration ofvarious chemical species. This data can then be interpreted as a“gradient map” of these species in a surface. In essence, HSI is amethod of “imaging spectroscopy” combining the chemical specificity ofspectroscopy with the spatial resolution of imaging.¹⁰ Light isseparated into hundreds of wavelengths using a spectral separator andcollected on a charge-coupled device (CCD) or a complementary metaloxide semiconductor (CMOS) sensor in much the same way that a picture istaken by an ordinary camera. Used for decades by the military, majorairborne applications now are also in mineral exploration andenvironmental and agricultural, assessments.^(11,12,13,14)

Biological tissues also have optical signatures that reflect theirchemical characteristics. The primary absorbers in tissue are oxy anddeoxy-hemoglohin, hemoglobin breakdown products (e.g. bilirubin androethemogiohin), melanin, (in skin), lipids and water. The in-vivoabsorption spectra of these compounds are well characterized.¹⁵ Bycomparing collected spectra to standard in-vivo absorption spectra,information about the type, location and relative concentration ofchromophores may be quantified.^(16,17) The use of MHSI in-vivo providesquantification of several parameters important in the assessment ofphysiology. These include oxygen delivery, oxygen extraction (correlatedwith tissue metabolism), total hemoglobin (correlated with perfusion)and water (correlated with tissue edema) with spatial patterns at thelevel of the microcirculation. Optical scattering also changes incancerous regions due to an increased number of cells with enlargednuclei. Scattering from mitotic spindles also increases due to the actthat a larger fraction of cells at any one time are undergoing mitosis.We have utilized the spectral and spatial features provided by MHSI todifferentiate diseased or cancerous tissue from normal tissue and todeliver information about the “functional anatomy” of themicrocirculation associated with local changes due to angiogenesisinfection, inflammation, ischemia, and the impact of local tumormetabolism and surrounding tissue response.¹⁸

HSI has been applied to biomedicine as a non-invasive diagnostic, MHSIis non-contact, camera-based, near-real time, and able to interlace withpotential patients a wide variety of settings, either in a diagnosticclinic or as a monitoring tool during surgery. MHSI has been appliedtoward the early determination of shock, the diagnosis of foot ulcersand foot microcirculation in diabetes, and in the evaluation ofresective surgery in breast cancer. MHSI can also utilize localinformation to evaluate systemic physiology and pathology and hasdemonstrated this ability in applications such as shock^(19,20,21) andprogression of diabetes²².

Significance of Hyperspectral Imaging in Cancer Diagnosis

HSI has the ability to transcend the limitations of the human eye anddeliver information present in the electromagnetic spectrum that isotherwise outside the range of our vision (e.g. IR, UV, etc.) and thatis beyond the level of our eye to discriminate (e.g. subtle wavelengthshifts corresponding to the shifting oxygenation state of hemoglobin).With respect to cancer diagnosis, development of a Hyperspectral CancerDetection (HCD) system provides quantitative diagnostic information at atime when clinical signs would be non-descript, inconclusive, or simplyabsent. The early determination of disease onset or progression or themore precise delineation of tumor margins or grade would clearly enhancethe power of intervention. As a novel non-invasive, near-real time tool,HCD has the potential to widely impact on the care of the cancerpatient.

The present invention uses real time intraoperative margin examinationto decrease the time, for completion of surgical, treatment and overallbreast cancer treatment. It significantly reduces cost related tooperative time and reoperative time as well reducing wait time betweenprocedures. It also reduces patient anxiety waiting to complete theirsurgical therapy.

MHSI also has potential for margin evaluation with many other tumortypes and thus can be applied to many endoscopic procedures, includingbut not limited to, laparescopy, colonoscopy, thoracoscopy, cystoscopy,hysteroscopy, bronchoscopy, and mediastinoscopy. Skin cancers, includingsquamous cell and basal cell carcinoma, are treated by local surgicalresection with frozen section analysis of the margins. Often, theresection of several additional margins is needed to completely removethe cancer and, daring this time, the surgeon, and patient are idle inthe operating room.

Gastrointestinal cancers, such as those of the esophagus and stomach,are known to infiltrate in submucosal planes, at some distance from themain mass. Achieving negative margins at the proximal and distal ends ofresection is a standard goal of surgery of these cancers. Real timerecognition of residual cancer cells at these margins would reduceoperative times. However, real time recognition, is not limited toresidual tumors, identification of tumors versus normal tissue cysts isanother embodiment of the present invention. The same is said forbiliary and pancreatic cancers.

Proper management of sarcomas is contingent upon achieving negativemargins. Many patients are found to have microscopically positivemargins despite what appears, grossly, to be an adequately wideexcision. It has been shown in numerous studies that liver resection forcolorectal metastases with positive margins is associated with a muchhigher recurrence rate and survival than when margins of resection arenegative, and in particular, exceed 1 cm.⁷

With pulmonary resection for lung cancer, clearance of the cancer at thebronchial stump margin is necessary. Negative margin excision iscritical in the surgical treatment of cancers of the head and neck. Atthe same time, tissue conservation is critical to reduce the potentialresulting deformity. Real time margin assessment would be invaluable.

Spectroscopy is used to assess optical properties (e.g., reflectance,absorbance, scattering) of materials of different composition and state.In medical applications, spectroscopy is widely used for in-vivomeasurements that assess the condition of a biological system, such asskin, tissue, or an organ. Once spectroscopy is performed simultaneouslyover a large area, it is called hyperspectral imaging. A simplifiedbiological multi-spectral imaging apparatus is a human eye that capturesreflected light at essentially three wavelength (red, green, and blue)and, once processed by a human brain, allows us to make conclusion aboutphysiological state (e.g., if a person is hot their skirt looks red).

The development of a hyperspectral imaging apparatus for medicalapplications allows, for expansion of the limitation of a human eye. Nowit is possible to acquire reflected light at hundreds of wavelengthsunder a minute over large areas. As a result, a three dimensional arrayof data (3D data cube) is obtained, containing an enormous amounts ofspatial and spectral information about the sample from which the datawas acquired.

Currently, there is no brain-like algorithm that can process vastamounts of spectral data, to facilitate the assessment of physiologicalconditions such as tissue viability or to make diagnoses or decisions inreal life situations such as during surgery or critical care in theemergency room. The volume of information contained in spectroscopicimages makes standard data processing techniques time consuming andcumbersome, furthermore, many techniques rely on matching to “learning”curves that require measurement of reference samples (controls) tocreate a library of spectra to facilitate the identification ofchemically related compounds.

The assessment of the metabolic state of tissue is important in areassuch as cancer detection, assessing surgical margins, screening for andmonitoring diabetes, and for monitoring shock. In many life situations(e.g. operating, emergency room or physician visit room), the assessmentof the metabolic state, or physiologic condition is necessary inreal-time. That requires a computerized algorithm that pulls out themost critical features from the vast amount of hyperspectral informationcaptured and presents results in an easily assessable color (orpseudo-color) image that can be interpreted by a person making decisionsin the time-pressing environments.

For example, in U.S. Pat. No. 5,782,770 Hyperspectral imaging methodsand apparatus for new-invasive diagnosis of tissue for cancer, byMooradian et al., an imaging device is described for capturinghyperspectral images of tissue and specifically for capturing,hyperspectral information that relates to the diagnosis of canceroustissue.

U.S. Pat. No. 6,640,130 Integrated imaging apparatus, by Freeman, etal., discusses application of a hyperspectral imager such as surgery,clinical procedures, tissue assessment, diagnostic procedures, healthmonitoring, and other medical valuations, especially when used incombination with other monitors of physiological assessment.

U.S. Pat. No. 6,750,964 Spectral imaging methods and systems, byLevenson and Miller, discusses general, methods of image processing(based on at least one of principal component analysis, projectionpursuit, independent component analysis, convex-hull analysis, andmachine learning) in application to hyperspectral cubes. In oneembodiment, uses reference and target samples to build training testsand therefore cannot be performed in near real-time without preparation.Another disadvantage is that '964 also requires prior library of spectrafor different tissue types, classifies each pixel based on correlationto the library samples.

U.S. Pat. No. 6,937,885, Multispectral/Hyperspectral Medical Instrument,by Lewis, et al., describes a medical hyperspectral/multispectral imagerfor assessing the viability of tissue including the detection ordiagnosis of cancer using organ or tissue specific diagnostic protocolmodules.

United States Patent Application No. 20010036304 Visualization andprocessing of multidimensional dam using prefiltering and sortingcriteria by Yang et al., describes a method for handling complexmultidimensional datasets generated by digital imaging spectroscopy thatallows organization and analysis applying software and computer-basedsorting algorithms. The sorting algorithms allow pixels or features fromimages and graphical data, to be rapidly and efficiently classified intomeaningful groups according to defined criteria.

U.S. Pat. No. 6,810,279 Hyperspectral imaging Calibration Device, byMansfield et al., describes hyperspectral imaging calibration devicesand methods for their use that generate images of three dimensionalsamples.

U.S. Pat. No. 6,640,132 Forensic Hyperspectral Instrument, by Freeman etal. describes portable imaging devices, such as hyperspectral imagingdevices, useful for forensic and other analysis, and methods for usingthese devices. Devices of '132 provide images and patterned data arraysrepresenting images in multiple discrete spectra that can then be summedor processed to allow for detection of patterns or anomalies in the datacollected. All of the aforementioned patents are hereby incorporated byreference.

It has surprisingly been discovered that tissues can be assessedspecifically for the detection of cancer or tumor beds during surgicalexcisions, hi the present invention, new methods and tissue specificalgorithms are presented that allow the assessment of tissue viabilityspecifically to the detection of cancer, or tumor beds during surgicalexcision. Cancer detection can include the detection of solid cancersand precancers, and blood borne cancers such as lymphoma. Cancerdetection can also include differentiating tumor from, normal, tissueand assessing the malignancy or aggressiveness or “grade” of tumorpresent. One problem is that often, (e.g. during surgery) it isnecessary to assess the tissue composition and oxygenation over a largearea and in real or near-real time.

MHSI solves that problem by processing the hyperspectral cubes in nearreal-time and presenting a high-resolution, pseudo-color image wherecolor varies with tissue type and oxygenation (viability). MHSI consistsof fast image processing steps that do not require prior knowledge ofthe tissue or its metabolic state, but that can also take additionalinformation from known tumor and tissue into account if so desired.

Another problem is the rapid screening of blood borne, cancer such aslymphoma in real-time without requiring blood draws and histologicassessment MHSI has the ability of assessing microvascular changes inskin due to lymphoma cell loading, MHSI also allows the quantitativemonitoring of cancer therapies as a means of optimizing treatment on anindividual basis or for the exploratory screening and optimization ofnew drugs. If is also possible that lymphoma or other tumor types may beassessed through the skin by evaluation of an underlying node or solidorgan. This may be useful in the staging of disease or in the monitoringof the results of a particular therapeutic regimen.

Yet another problem is that the results of the analysis have to bepresented in an easily accessible and interpretable form. MHSI deliversresults in a very intuitive form by pairing the MHSI pseudo-color imagewith the high quality color picture composed from the same hyperspectraldata. The identification and assessment of the region of interest (ROI)is easily achieved by flipping between color and MHSI images, andzooming onto the ROI. The images can be seen on a computer screen orprojector, and/or stored and transported as any other digitalinformation, and/or printed out. The MHSI image preserves the highresolution of the hyperspectral imager and therefore can be improvedwith the upgraded hardware.

A fourth problem is that due to the complexity of the biological system,medical personnel want to have as much information as possible about agiven case in order to make the most-reliable diagnose. MHSI providesadditional information to the doctor that is not currently available andcan be used along with other clinical assessments to make this decision.MHSI provides images for further analysis by the human; initially it isnot an artificial intelligence decision maker and you would not want torely directly on the software to make such an important decision,however as more information is gathered, a spectral library compiled andtechniques refined, MHSI has the capabilities necessary to be a truediagnostic device.

Additionally, MHSI transcribes vast 3D spectral information into oneimage preserving biological complexity via millions of color shades. Theparticular color and distinct shape of features in the pseudo-colorimage allow discriminate between tissue types such as tumor, connectivetissue, muscle, extravasaled blood, and blood vessels. MHSI also allowsthe near-real time differentiation of tumor grade that will be useful inmaking appropriate medical decisions.

Yet another problem, is quantifying cancer therapies in order to measurethe effectiveness of new therapeutic agents or procedures. MHSI can beused to quantify disease progression in order to identify newtherapeutic agents and to develop individual therapeutic regimentsdepending on how the subject responds to therapy.

MHSI's main purposes include 1) expanding human eye capabilities beyondthe ordinary; 2) expanding the human brain capabilities by pro-analyzingthe spectral characteristics of the observable subject; and 3)performing these tasks with real or near-real time data acquisition. Theaim of the algorithm is facilitate the human to diagnose and assess thecondition of the observable subject.

MHSI is successful because it is complete, using the spectral data ofreflected electro-magnetic radiation (ultraviolet—UY, visible, nearinfrared—NIR, and infrared—IR), and since different types of tissuereflect, absorb, and scatter light differently, in theory thehyperspectral cubes contains enough information to differentiate betweentissue types and conditions. MHSI is robust since it is based on a fewgeneral properties of the spectral profiles (slope, offset, and ratio)therefore is pretty flexible with respect to spectral, coverage and isnot sensitive to a particular light wavelength, MHSI is fast, because ituses fast image processing techniques that allow superposition ofabsorbance, scattering, and oxygenation information in one pseudo-colorimage.

The simplicity of image processing techniques allow for the display ofresults in real-to-near-real time. MHSI is easily interpretable since itoutputs image where color changes according to different tissue types orcondition, but the distinction is not a yes/no type, MHSI color schemeallows surgeon to differentiate between different tissue types, inaddition, the color and the shape of structures depict differentcomposition and level of viability of the tissue. For example, tumorappears to have an atypical, color and appears as a rounded, almostsolid color mass. The blood vessels are rather differentiated by theirshape as linear and curvilinear structures (wiggly strings) than bytheir color per se; the exact color of the vascular structures dependson the blood oxygenation.

Initially, the algorithm needs a person, to conclude that the tissue, istumor or normal. In another embodiment, a particular color code containsadequate information for diagnosis and are presented as such. Initeration, MHSI by itself is not a definite decision, making algorithm;it is a fool that a medical professional can use in order to give aconfident diagnosis, in iteration MHSI contains a decision makingalgorithm that provides the physician with a diagnosis.

A portable hyperspectral imaging apparatus according to an embodiment ofthe invention is depicted in FIG. 1. Portable apparatus 10 weighs lessthan 100 pounds, preferably less than 25 pounds, and more preferablyless than 10 pounds. Preferably, the portable apparatus may be batteryoperated or more preferably, may have a connector adapted to connect toan existing power source.

Portable apparatus 10 comprises an optical acquisition system 36 and adiagnostic processor 38. Optical acquisition system 36 comprises meansto acquire broadband data, visible data, ultraviolet data, infra-reddata, hyperspectral data, or any combination thereof. In a preferredembodiment, optical acquiring means comprises a first-stage imagingoptic 40, a spectral separator 42, a second-stage optic 44, and animaging sensor 46. Alternatively, optical acquiring means may be anyacquisition system suited for acquiring broadband data, visible data,ultraviolet data, infra-red data, hyperspectral data, or any combinationthereof. Preferably, one or more polarizers 41, 43 are included in theacquisition system to compile the light into a plane of polarizationbefore entering the imaging sensor.

If the spectral separator 42 does not internally polarizes the light,the first polarizer 43 is placed anywhere in the optical path,preferably in front of the receiving camera 46. The second polarizer 41is placed in front of illuminating lights (20) such that the incidentlight polarization is controlled. The incident light is crossedpolarized with the light recorded by the camera 46 to reduce specularreflection or polarization at different angles to vary intensity of thereflected light recorded by the camera.

The illumination is provided by the remote light(s) 20, preferablypositioned around the light receiving opening of the system. The lightcan fee a circular array of focused LED lights that emit light at theparticular wavelengths (or ranges) that are used In the processingalgorithm, or in the ranges of wavelengths (e.g., visible and/ornear-infrared). The circular arrangement of the light sources provideseven illumination that reduces shadowing. The light wavelengthselectivity reduces effect of the observation on the observing subject.

Although the preferred embodiment describes the system, as portable, anon-portable system may also be utilized. Preferably, an optical bead ismounted to the wail of the examination, more preferably, an overheadlight structure is located in the operating room, or more preferably,the system has a portable table with an observational window overlookingthe operating site.

The first-stage optic receives light collected from a tissue samplethrough a polarizer and focuses the light onto the surface of thespectral separator. Preferably, the spectral separator is a liquidcrystal tunable filter (LCTF). LCTF 42 is a programmable filter thatsequentially provides light from selected wavelength bands with small(for example, 7-10 nm) bandwidth from the light collected from thesample. Second-stage optic 44 receives the narrow band of light passingthrough the spectral separator and focuses the light onto the imagesensor 46. The image sensor is preferably, although not necessarily, atwo-dimensional array sensor, such as a charge-coupled device array(CCD) or CMOS, which delivers an image signal to the diagnosticprocessor 38.

Diagnostic processor 38 includes an image acquisition interface 50, thathas an input responsive to an output of the image sensor 46 and anoutput provided to a general-purpose operating module 54. Thegeneral-purpose operating module includes routines that perform imageprocessing, and that operates and controls the various parts of thesystem. The general-purpose operating module also controls the lightsource(s) (e.g. LED array) allowing for switching on and off duringmeasurement as required by the algorithm. The general-purpose operatingmodule has control output provided to a filter control interface 52,which in turn has an output provided to the spectral separator 42. Thegeneral-purpose operating module also interacts with a number ofdiagnostic protocol modules 56A, 56B, . . . 54N, and has an outputprovided to a video display. The diagnostic process includes specialpurpose hardware, general-purpose hardware with special-purposesoftware, or a combination of the two. The diagnostic processor alsoincludes an input device 58, which is operatively connected to thegeneral-purpose operating module. A storage device 60 and printer 62also are operatively connected to the general-purpose operating module.

In operation, a portable or semi-portable apparatus is employed near atarget, e.g., breast tumor resection bed or general area of interest. Anoperator begins by selecting a diagnostic protocol module using theinput device. Each diagnostic protocol module is adapted to detectparticular tissue characteristics of the target. In an alternativeembodiment, the apparatus may contain, only one diagnostic moduleadapted for general medical diagnosis.

Diagnostic processor 38 responds to the operator's input by obtaining aseries of transfer functions and an image processing protocol and animage processing protocol from the selected diagnostic protocol module56. The diagnostic processor provides the filtering transfer functionsto the spectral separator 42 via its filter control interface 52 andthen instructs the image acquisition interface 50 to acquire and storethe resulting filtered image from the image sensor 46. Thegeneral-purpose operating module 54 repeats these filtering andacquiring steps one or more times, depending on the number of filtertransfer functions stored in the selected diagnostic protocol module.The filtering transfer functions can represent bandpass, multiplebandpass, or other filter characteristics and can include wavelengths inpreferably the UV, preferably the visible, preferably the NIR andpreferably, the IR electromagnetic spectrum.

In a preferred embodiment, the light source delivering light to thetarget of interest can be filtered as opposed to the returned lightcollected by the detector. Thus, a tunable source delivers theinformation. Alternatively, both a tunable source and a tunable detectormay be utilized. Such tuning takes the form of LCTF, acousto-opticaltunable filter (AOTF), filter wheels, matched filters, diffractiongratings or other spectral separators. The light source may be a fiberoptic, but is preferably a light emitting diode (LED).

The unique cooling illumination provided by the LED prevents overheatingof skin which may result in poor imaging resolution. Preferably, the LEDprovides sufficient light while producing minimal or no increase in skintemperature. This lighting system in combination with the polarizerallows adequate illumination while preventing surface glare frominternal organs and overheating of skin.

Once the image acquisition interface 50 has stored images for all of theimage planes specified by the diagnostic protocol chosen by theoperator, the image acquisition interface begins processing these imageplanes based of the image processing protocol from the selecteddiagnostic protocol module 56N. Processing operations can includegeneral image processing of combated, images, such as comparing therelative amplitude of the collected light at different wavelengths,adding amplitudes of the collected light at different wavelengths, orcomputing other combinations of signals corresponding to the acquiredplanes. The computed image is displayed on the display 12. Otherpreferred embodiments include storing the computed image in the storagedevice 60 or printing the computed image out on printer 62.

In an alternative embodiment, diagnostic protocol modules 56, printer62, display 12, or any combination thereof, may be omitted from portabledevice 10. In this embodiment, acquired images are stored in storagedevice 60 daring the medical procedure. At a later time, these imagesare transferred via a communications link to a second device or computerlocated at a remote location, for example, hospital medical records, forbackup or reviewing at a later time. This second device can have theomitted diagnostic protocol modules, printer, display, or anycombination thereof. In another embodiment, the stored images aretransferred from portable device 10, located in the clinic, via acommunications link to a remote second device in real time.

In a preferred embodiment the system has facility to project real-timehyperspectral data onto the operation field, region of interest, orviewing window positioned above the operating site. The projectedinformation has precise one-to-one mapping to the illuminated surface(e.g. wound, operating surface, tissue) and provides surgeon withnecessary information in efficient and non-distractive way. Whenprojected onto an overhang viewing window, the images (real-color and/orpseudo-color) can be zoomed in/out to provide variable magnification.This subsystem consists of the following elements; 1) an image projectorwith field-view precisely co-aligned with the field-of-view of thehyperspectral imager, 2) a miniature remote control device which allowssurgeon to switch projected image on and off without turning fromoperation table and change highlight structure and/or translucency onthe projected, image to improve visibility of the features of interestas well as projected, image brightness and intensity, 3) a real-timedata processing package which constructs projected image based onhyperspectral data and operator/surgeon input, and 4) an optionalviewing window positioned above the operating site that is translucentfor real observation or opaque for projecting pseudo-color solution orhigher resolution images.

To achieve precise co-registration between hyperspectral image andoperating surface, the system performs self-alignment procedure beforeor during the operation as necessary. The system projects a sequence ofcalibration pattern on the operating surface using projector and readsthem using hyperspectral imaging system. Calibration software processesacquired data and stores them. Processed data are further used byprojection system to achieve high-precision mapping to operating surfaceand compensate for surface relief.

Devices of the present invention allow for the creation and uniqueidentification of patterns in data that highlight the information ofinterest. The data sets in this case may be discrete images, eachtightly bounded in spectra that can then be analyzed. This is analogousto looking at a scene through various colored lenses, each filtering outall but a particular color, and then a recombining of these images intosomething new. Such techniques as false color analysis (assigning newcolors to an image that don't represent the true color but are anartifact designed to improve the image analysts by a human) are alsoapplicable. Optionally, optics can be modified to provide a zoomfunction, or to transition from a micro environment to a macroenvironment and a macro environment to a micro environment. Further,commercially available features can be added to provide real-time ornear real-time functioning. Data analysis can be enhanced bytriangulation with two or more optical acquisition systems. Polarizersmay be used as desired to enhance signatures for various targets.

In addition to having the ability to gather data, the present inventionalso encompasses the ability to combine the data in various mannersincluding vision fusion, summation, subtraction and other, more complexprocesses whereby certain unique signatures for information of interestcan be defined so that background, data and imagery can be removed,thereby highlighting features or information of interest. This can alsobe combined with automated ways of noting or highlighting items, areasor information of interest in the display of the information.

The hyperspectrally resolved image in the present invention is comprisedof a plurality of spectral bands. Each spectral hand is adjacent toanother forming a continuous set. Preferably, each spectral band has abandwidth of less than 50 nm, more preferably less than 30 nm, morepreferably less than 20 nm, more preferably, from about 20-40 nm, morepreferably, from about 20-30 nm, more preferably, from about 10-20 nm,more preferably from about 10-15 nm, and more preferably from about10-12 nm.

It is clear to one skilled in the art that there are many uses for amedical hyperspectral imager (MHSI) according to the invention. The MHSIoffers the advantages of performing the functions for such uses faster,more economically, and with less equipment and infrastructure/logisticsentailed than other conventional techniques. Many similar examples canbe ascertained by one of ordinary skill in the art from this disclosurefor circumstances where medical personal relies on their visual analysisof the biological system. The MHSI acts like “magic glasses” to helphumans to see inside and beyond.

Algorithm Description

The embodiment of the cancer detecting algorithm involves the followingsteps:

1. Preprocess the HSI data. Preferably, by removing background radiationby subtracting the calibrated background radiation from each newlyacquired image while accounting for uneven light distribution bydividing each image by the reflectance calibrator image and registeringimages across a hyperspectral cube.

2. Build a color-photo-quality visual image. Preferably, byconcatenating three planes from the hyperspectral cube at thewavelengths that approximately correspond to red (preferably in therange of about 580-800 nm, more preferably in the range of about 600-700nm, more preferably in the range of about 625-675 nm and more preferablyat about 650 nm), green (preferably in the range of about 480-580 nm,more preferably in the range of about 500-550 nm, more preferably in therange of about 505-515 nm, and more preferably at about 510 nm), andblue (preferably in the range of about 350-490 nm, more preferably inthe range of about 400-480 nm, more preferably in the range of about450-475 am, and more preferably at about 470 nm) color along the thirddimension to be scaled for RGB image.

3. Define a region of interest (ROI). Preferably, where the solution isto be calculated unless the entire field of view to be analyzed.

4. Convert all hyperspectral image intensities into units of opticaldensity. Preferably, by taking the negative logarithm of the decimalbase. FIG. 2 shows examples of spectra taken from single pixels atdifferent tissue sites within an image. Tissue sites include connectivetissues, oxygenated tissues, muscle, tumor, and blood.

5. Decompose the spectra for each pixel (or ROI averaged across severalpixels). Preferably, decompose into several independent components, morepreferably, two of which are oxyhemoglobin and deoxyhemoglobin,

6. Determine three planes for RGB pseudo-color image. Preferably,determine by using characteristic features of the spectra. Preferably,the red plane is a slope coefficient for the blue portion of the spectra(wavelengths shorter than about 500 nm) at each pixel; the green andblue planes are the offset and the slope coefficients for the redportion of the spectra (starting from about 640 nm and longer) at eachpixel. More preferably, coefficients for oxy and deoxy components (orother components of spectral decomposition), or their combination may beused to define the red, green, and blue planes. More preferably, acombination of the spectral images at different wavelengths, for examplethe ratio (or difference) between a wavelength in the red region(preferably in the range of about 580-800 nm, more preferably in therange of about 600-700 nm, more preferably in the range of about 625-675nm and more preferably at about 650 nm) and a wavelength in the yellow(preferably in the range of about 550-580 ran, more preferably in dierange of about 555-575 nm, more preferably in the range of about 560-570nm, and more preferably at about 565 nm) or in the orange (preferably inthe range, of about 580-615 nm, more preferably in the range of about585-610 nm, more preferably in the range of about 590-605 nm, morepreferably in the range of about 595-605 nm, and more preferably atabout 600 nm) regions may be used.

7. Determine a sharpness factor plane, preferably, by using acombination of the images at different wavelengths. In one embodiment,taking a ratio of a yellow plane (preferably in the range of about550-580 nm, more preferably in the range of about 555-575 nm, morepreferably in the range of shout 560-570 nm, and more preferably atabout 565 nm) to a green plane (preferably in the range of about 480-580nm, more preferably in the range of about 500-550 nm, more preferably inthe range of about 505-515 nm, and more preferably at about 510 nm) wasused. In another embodiment, a combination of oxyhemoglobin anddeoxyhemoglobin spectral components as a sharpness factor plane wasused. In yet another embodiment, a combination of the spectral images atdifferent wavelengths, for example, the ratio (or difference) between awavelength in the red region (preferably in fee range of about 580-800nm, more preferably in the range of about 600-700 nm, more preferably inthe range of about 025-675 nm and more preferably at about 650 nm) and awavelength in the yellow (preferably in the range of about 550-580 nm,more preferably. In the range of about 555-575 nm, more preferably inthe range of about 560-570 nm, and more preferably at about 565 nm) orin the orange (preferably in the range of about 580-615 nm, morepreferably in the range of about 85-610 nm, more preferably in the rangeof about 590-605 nm, more preferably in the range of about 595-605 nm,and more preferably at about 600 nm) regions was used,

8. Convert RGB Image to hue-saturation-value/intensity (HSV/I) image andscale the value (or intensity) plane with the sharpness factor plane.Convert HSV/I back to RGB image.

9. Remove outliers, in the resulting image, defining an outlier as colorintensity deviating from a typical range beyond certain number ofstandard deviations, preferably three. Stretch the resulting image tofill entire color intensity range, e.g. between 0 and 1 for a doubleprecision image.

10. Display ROI in pseudo-colors. Preferably, in combination with thecolor photo image of die subject, or preferably, in addition to thecolor photo image of the subject, or more preferably, by projecting thepseudo-color image onto the observed surface. FIG. 3, Panel C shows anillustrative example of a MHSI image. The algorithm used for this imageis based on analysts of the absorption spectra in the visible lightwavelength range (see example of spectra in FIG. 3). Subtle changes insmall blood vessel oxygen saturation are clearly demonstrated. Highlyoxygenated tissue, appears light on the MHSI images whereasextravasating blood is darker. The location of a residual tumor can beconveyed to the surgeon by projecting the pseudo-color image or somevariance such as a binary image of the residual tumor directly onto thetumor bed. Additional information can be conveyed through imagesportraying the oxyhemoglobin, deoxyhemoglobin, slope and offsetcoefficients, or any linear or nonlinear combination such as theoxyhemoglobin to deoxyhemoglobin ratio.

11. Characterize the metabolic state of the tissue of interest (e.g.tumor grade, hematoma age, connective tissue density, etc). Preferably,by using the saturation and/or intensity of the assigned color andprovide a qualitative color scale bar.

As is clear to a person of ordinary skill in the art one or more of theabove steps in the algorithm can be performed in a different order oreliminated entirely and still produce adequate and desired results.Preferably, the set of instructions includes only the steps ofpreprocessing, the hyperspectral information, building a visual image,using the entire field of view, converting all hyperspectral imageintensities into units of optical density by taking a negative logarithmof each decimal base, and characterizing a metabolic state of the tissueof interest. More preferably, the set of instructions comprisespreprocessing the hyperspectral information, defining a region ofinterest of the tissue, and characterizing a state of the tissue ofinterest.

Another preferred embodiment entails reducing the hyperspectral data inthe spectral dimension into a small set of physiologic parametersinvolves resolving the spectral images into several linearlyindependent, images (e.g. oxyhemoglobin, deoxyhemoglobin, an offsetcoefficient encompassing scattering properties and a slope coefficient)in the visible regime. Another embodiment determines four images (e.g.oxyhemoglobin, deoxyhemoglobin, offset/scattering coefficient, and waterabsorption) in the near infrared region of the spectrum. As an examplefor the visible region of the spectrum, linear regression fitcoefficients c₁, c₂, c₃ and c₄ will be calculated for reference oxy-Hb,deoxy-Hb, and MS spectra, respectively, for each spectrum (Sij) in animage cube:

{right arrow over (S)} _(ij) =∥c ₁ {right arrow over (OxyHb)}+c ₂ {rightarrow over (DeoxyHb)}+c ₃{right arrow over (Offset+c ₄Slope)}∥₂

Individual images of the oxyhemoglobin and deoxyhemoglobin components,the slope and offset or any combination, linear or nonlinear, of theseterms, for example the oxy- to deoxyhemoglobin ratio, can be presentedin addition to producing the pseudo-colored image to the user. Thismethod is particularly useful for assessing microvascular changes intissue such as in the lymphoma application.

During breast cancer surgery, die surgeon opens up the surgical site andvisually observes the area where blown or suspected tumor grows. Thehyperspectral imaging system, will also acquire data, and preferablywithin a minute produce color and pseudo-color images of the site inquestion, where tissue is colorized according to its composition andviability (e.g., oxygen saturation) with tumors specificallyhighlighted. The medical team visually evaluates the extent of thedisease taken into account the information presented in thehyperspectral image, then they decide what areas are affected by thetumor and to what degree. The surgeon excises the suspicions tissue.FIG. 4 shows an example of a tumor with MHSI image prior to excision.

Following excision, standard practice would involve taking samples ofthe surrounding tissue typically at the margins of the tumor resectionsite, which are sent for histopathologic evaluation to assess thepresence of residual cancerous cells. This process is time consuming andcan take up to two-hours. “Frozen sections” of tissue from the marginsare performed at randomly chosen sites and a preliminary diagnosis ofpresence or absence of residual tumor is made. If residual tumor isfound by this method, resection of additional tissue is undertaken atthe time of original surgery. The resected margins are also placed informaldehyde and sent for “permanent section” which is more likely toprovide an accurate diagnosis. If tumor is found on permanent section,when it was not found on frozen section, the patient is brought back tothe operating room for additional surgery.

This embodiment describes a method where within two minutes a set ofhyperspectral data are collected from the region around a resected tumorand MHSI images produced that evaluate the presence of cancer in theexposed surfaces of the rumor resection bed. If there is residual tumorleft or uncovered, the surgery team will be able to detect the latterwithin 2 minutes of resection, and either send targeted pieces of tissueto pathology to confirm the diagnosis of residual tumor prior toexcising additional tissue or continue excising until the tumor bed isclean. The MHSI is capable of examining the entire excisional wound bedwhich may be important for locating small, nests of tumor cells (under0.5 mm) unlike the random. 4- or 5-point biopsy approach. Such residualtumor will be detected at once with MHSI at the time of operation. FIG.5 shows an example of this concept. Including MHSI images of the tumor,prior to uncovering, after uncovering, after the initial resection, andfrom a clean wound bed.

Additionally, evaluation of lymph nodes for tumor Involvement at thetime of surgery or potentially through the skin is possible with similartechniques. Similar techniques are applied to the assessment ofresection margins with other cancers such as gastrointestinal (stomach,colon, etc) gynecologic (cervical, ovarian, etc) urologic (prostate,renal cell) and other forms of cancer.

HSI can be placed in similar probes endoscopically, including but notlimited to, laparoscopically, thoracoscopically, cystoscopically,hysteroscopically, bronchoscopically, and mediastinoscopically to assesspresence of tumor, adequacy of surgical resection or nodal orintracavitary spread.

Similar techniques are used endoscopically for the primary detection oftumors of the GI tract and to define adequacy of resection orrecurrence. The pseudocolor images delivered would facilitate easy andrapid tumor identification, classification of polyps, evaluation ofBarrett's esophagus or identification and evaluation of both surface andsubmucosal processes.

MHSI Images Allow Tissue Identification, Tumor Grade Separation, andIn-Vivo Histology

FIG. 6 shows illustrative MHSI Images (color and pseudo-color images)that distinguishes hematoma from tumor. Hematoma and extravasated blood(red/pink/orange in left panel) are often visually indistinguishablefrom residual tumor to the eye of the surgeon, or in a simple colorpicture, whereas in the MHSI pseudo-color image (right panel) blood isseen as darker shading, oxygenated tissue as lighter shading, andresidual tumor as moderate shading masses. Examples of tissue typesidentified in this image include muscle, residual tumor, connectivetissue, extravasated blood, and a hematoma.

Since pseudo colors in the MHSI images are determined from the tissueabsorption spectra, any variations in the metabolic state (no matter howsmall) will be reflected through gradation in the color. The tumorsgenerally are graded by histo-pathologists according to the followingclassification:

0 1 2 3 4 normal tissue benign tumor intraductal papillary and papillary& carcinoma cribriform cribriform carcinoma carcinoma with invasionand/or comedo carcinoma areas

FIG. 7 shows representative images for each grade, starting from normaltissue (left image) and progressing to grade 4 (right image).

At high resolution, MHSI presents structural information that is similarto information gather from histologic slides. FIG. 8 depicts an imagefrom a tumor in situ (4×3 cm) that, was collected by the surgeon.Resected tumor and surrounding tissue (5×7 mm) was evaluated byhistopathology after biopsy. Microscopic images with further resolutionare displayed showing the histologic features mirrored in MHSI image.Characteristics of the invasion and the invasiveness of a tumor mayactually be better appreciated in vivo by MHSI man by means of the invitro histology previously required and may provide additional,information which are added to or are supplanted traditionalhistopathology in terms of defining prognosis and directing therapy.

MHSI Screening and Assessment of Lymphoma

A similar diagnostic algorithm based from MHSI images can be describedfor the screening and assessment of lymphoma. Lymphoma havingcirculating leukemic cells presents with unique symptoms, including theleukemic load or amount of leukemic cells in blood, systemicmicrovascular changes caused by leukemic cell clumping, and systemicdevelopment of leukemic tumor nodules, particularly in the lymph nodesand spleen. MHSI can be used to identify these changes to enablescreening for the disease and monitoring the progression of the disease.It is envisioned that disease progression can be monitored duringtherapy such that the management can be tailored for the individual'sresponse to therapy.

Uses of MHSI for lymphoma include imaging the lymph nodes or spleenvisualized through the skin, or during endoscopic or open surgicalprocedures in order to assess the progression of the disease byevaluating the size of the spleen and number or density of rumornodules. FIGS. 9-11 show MHSI taken through the skin of a mouse with andwithout lymphoma. Standard non-MHSI images along with MHSI images ofoxyhemoglobin, deoxyhemoglobin, total hemoglogin and oxygen saturationare presented from a disease and normal mouse. The change in the size ofthe spleen is noted in the color image as well as an increase innodularity in the pseudo-color image of the disease mouse when comparedto the normal control mouse.

Disease progression can also be monitor with MHSI by visualizingmicrovascular changes in the skin and eye, or other vascular sites suchas the ear, lips, oral mucosa, and tongue. FIGS. 12-14 show examples ofmicrovascular changes noted from the ear and eye of a diseased andnormal mouse. Standard non-MHSI images along with MHSI images ofoxyhemoglobin, deoxyhemoglobin, total hemoglogin and oxygen saturationare presented from a disease and normal mouse. A marked decrease inoxyhemoglobin and oxygen saturation in eye, ear skin and blood vesselsin lymphomatous mouse is seen when compared to a normal mouse. There aresimilar but much less marked differences in deoxy and total hemoglobinimages. These changes are consistent with both a decrease in red cellvolume and in a decrease in flow with greater oxygen extraction, by thetissue.

It is also envisioned that disease progression using similar algorithmscan be monitored in small and large animals as a means for developingand optimising new therapeutic agents for curing this disease.

The evaluation of other leukemias and hematogenous cancers as well asinvolvement of lymph nodes, solid organs and other tissue by otherlymphomas, other cancers and other tumors can also be undertaken bysimilar techniques and will be apparent to those skilled in the art.

Other embodiments and uses of the invention will be apparent to thoseskilled in the art from consideration of the specification and practiceof the invention disclosed, herein. All references cited herein,including all publications, U.S. and foreign patents and patentapplications, are specifically and entirely incorporated by reference.It is intended, that the specification and examples be consideredexemplary only.

REFERENCES CITED

-   ¹CA: A Cancer Journal for Clinicians; Cancer Statistics, 2005,    55:10-30 January/February.-   ²National Cancer Database-   ³Fisher B. Anderson S. Bryant J. Margolese E C. Deutseh M. Fisher    E R. Jeong J H. Wolmark N. Twenty-year follow-up of a randomized    trial comparing total mastectomy, lumpectomy, and lumpectomy plus    Irradiation for die treatment of invasive breast cancer. New England    Journal of Medicine, 347(16):1233-41, 2002 Oct. 17-   ⁴Singletary, S. Eva, “Surgical margins in patients with early stage    breast cancer treated wife breast conservation therapy” The American    Journal of Surgery 2002; 184:383-393,-   ⁵Frazier T G, Wong R W, Rose D: Implications of accurate pathologic    margins in the treatment of primary breast cancer. Arch Surg    124:37-38, 1989-   ⁶Hansen N M, Grube B J, Giuliano, A E The time has come to change    the algorithm for the surgical management of early breast cancer.    Archives of Surgery. 137(10): 1131-5, 2002 October.-   ⁷Gibson G R. Lesnikoski B A. Yoo J. Mott L A. Cady B. Barm R J Jr. A    comparison of ink-directed and traditional whole-cavity re-excision    for breast lumpectomy specimens with positive margins. Annals of    Surgical Oncology, 8(9):693-704, 2001 October.-   ⁸National Cancer Institute    [http://www.nci.nm.gov/cancer_information].-   ⁹Weeks J C, Yeap B Y, Canellos G P, Shipp M A. Value of follow-up    procedures in patients with large cell lymphoma, who achieve a    complete remission. J. Clin. Oncol. 1991; 9:1190-4203.-   ¹⁰Colamsso P, Kidder L H, Levin I W, et al Infrared spectroscopic    imaging: from planetary to cellular systems, Appl Spectrosc 1998;    52; 106A-120A. Treado P J, Morris M B. Appl Spectrosc Rev 1994;    29:1-38.-   ¹¹Riaza, A, Sirohl, P, Beisl, U, Hausold, A, Muller, A., Spectral    mapping of rock weathering degrees on granite using hyperspectral    DAIS 7915 spectrometer data, IJ of Applied Earth Observation and    Geoinformation, January 2001.-   ¹²Thenkabail, P S, Smith, R E, De Pauw, E Hyperspectral Vegetation    Indices and Their Relationships with Agricultural Crop    Characteristics Volume 71, Issue 2, February 2000, Page 158-182.-   ¹³Bowles, J H. J A. Antoniades, M M. Baumbaok, J M. Grossmann, D.    Haas, P J Palmadesso and J. Stracka. (1997). Real-time analysis of    hyperspectral data sets using NRL's ORASIS algorithm. Proc. SPIE    Vol. 3118, p, 38-45.-   ¹⁴Tran C D. Development and analytical applications of multispectral    imaging techniques: an overview. Fresenius J Anal Chem 2001    February; 36913-4):313-9.-   ¹⁵Sowa M G, TR. Mansfield. M. Jackson, J. C. Docherty, R.    Deslauriers, find H. H. Mantsch. “FT-IR/NIR Assessment of Ischemic    Damage is the Rat Heart” Mikrochimca Acta [suppl.], 1997, 14,    451-453.-   ¹⁶Doornbos R M, Lang R, Aalders M C, Cross F W, Sterenborg H J. The    determination of in vivo human tissue optical properties and    absolute chromophobe concentrations using spatially resolved    steady-state diffuse reflectance spectroscopy. Phys Med Biol 1999    April; 44(4):967-81.-   ¹⁷Zonios G, Bykowski J, Kollias N. Skin melanin, hemoglobin, and    light scattering properties can be quantitatively assessed in vivo    using diffuse reflectance spectroscopy. J Invest Dermatol 2001    December; 117(6): 1452-7.-   ¹⁸Freeman, et al. Medical Hyperspectral Imaging (MHSI) of    1,2-dimethylbenz(a)-anthracene (DMBA) Induced Breast Tumors In Rats.    San Antonio Breast Cancer Symposium, 2004.-   ¹⁹Cancio, L. C; Brand, D; Kerby, J; Freeman, J; Hopmeier, M; and    Mansfield, J.-   R. “Visible Hyperspectral Imaging; Monitoring the Systemic Effects    of Shock and Resuscitation.” Proc SPIE 4614:155), 2002.-   ²⁰Panasyak S V, Freeman I E, Cooke W, Hopmeier M, Converimo V.    Initial Demonstration in Human Subjects of Medical Hyperspectral    Imaging (MHSI) as a Novel Stand-Off Non-Invasive Method for    Diagnosing and Measuring Hemodynamic Collapse. Accepted for American    Association of Shock and Trauma Annual Meetings 2005.-   ²¹Freeman J E, Panasyuk S V, Hopmeier M J, Lew R A, Batchinski A,    Cancio L C. Evaluation of New Methods of Hyperspectral Image    Analysis for the Diagnosis of Hemorrhagic Shock. Accepted for    American Association of Shock and Trauma Annual Meeting, 2005.-   ²²Greenman R L, Panasyuk S, Wang X, Lyons T E, Dinh T, Langoria L,    Giurini M, Freeman J, Khaodhiar L, Veves A. Early changes in the    skin microcirculation and muscle metabolism of the diabetic foot    Lancet 2005; 360: 1711-17.

1. A multispectral or hyperspectral medical imaging system comprising: aplurality of LED lights, each respective LED light in the plurality ofLED lights configured to emit radiation at a particular spectral band ina plurality of spectral bands used by a diagnostic protocol module; afirst stage optic configured to receive radiation projected, from one ormore respective LED lights in the plurality of LED lights, off of an invivo tissue located at a region of interest on a subject; one or morepolarizers in optical communication with the first stage optic; animaging sensor for recording an image of the radiation projected off ofthe in vivo tissue; a diagnostic protocol module adapted to detect aparticular characteristic of the in-vivo tissue; a diagnostic processorconfigured to: switch one or more respective LED light in the pluralityof LED lights off and on based on the diagnostic protocol module,instruct the imaging sensor to record a plurality of images of theregion of interest based on the diagnostic protocol module, eachrespective image in the plurality of images corresponding to radiationemitted at a particular spectral band by one or more respective LEDlight in the plurality of LED lights, obtain a multispectrally orhyperspectrally resolved image based on the plurality of images, andobtain a pseudo-color image of the region of interest based on themultispectrally or hyperspectrally resolved image, the pseudo-colorimage enhancing the visibility of the particular characteristic of thein vivo tissue present in the region of interest; and a projectionsubsystem configured to project the pseudo-color image onto the regionof interest, wherein the projection subsystem comprises an imageprojector with a field-of-view co-aligned with a field of view of thefirst stage optic.
 2. The multispectral or hyperspectral medical imagingsystem of claim 1, wherein the imaging system is portable.
 3. Themultispectral or hyperspectral medical imaging system of claim 2,wherein the imaging system weighs less than 25 pounds.
 4. Themultispectral or hyperspectral medical imaging system of claim 1,wherein the plurality of LED lights is configured in a circular arrayabout the first stage optic.
 5. The multispectral or hyperspectralmedical imaging system of claim 1, wherein at least one respective LEDlight in the plurality of LED lights is configured to emit radiation inthe visible spectrum.
 6. The multispectral or hyperspectral medicalimaging system of claim 1, wherein at least one respective LED light inthe plurality of LED lights is configured to emit radiation in thenear-infrared spectrum.
 7. The multispectral or hyperspectral medicalimaging system of claim 5, wherein at least one respective LED light inthe plurality of LED lights is configured to emit radiation in thenear-infrared spectrum.
 8. The multispectral or hyperspectral medicalimaging system of claim 1, further comprising a first radiation filterplaced in front of a respective LED light in the plurality of LED lightsand a second, matching radiation filter, placed in optical communicationwith the imaging sensor.
 9. The multispectral or hyperspectral medicalimaging system of claim 1, wherein a first polarizer in the one or morepolarizers is placed in front of a respective LED light in the pluralityof LED lights and a second polarizer in the one or more polarizers isplaced in optical communication with the first stage optic.
 10. Themultispectral or hyperspectral medical imaging system of claim 1,wherein the imaging sensor is configured to detect radiation in thevisible spectrum.
 11. The multispectral or hyperspectral medical imagingsystem of claim 1, wherein the imaging sensor is configured to detectradiation in the near-infrared spectrum.
 12. The multispectral orhyperspectral medical imaging system of claim 10, wherein the imagingsensor is configured to detect radiation in the near-infrared spectrum.13. The multispectral or hyperspectral medical imaging system of claim1, wherein the diagnostic processor is further configured to provide anoutput to a video display.
 14. The multispectral or hyperspectralmedical imaging system of claim 13, wherein the video display iscontained within the multispectral or hyperspectral medical imagingsystem.
 15. The multispectral or hyperspectral medical imaging system ofclaim 1, further comprising a communication interface in electricalcommunication with the diagnostic processor.
 16. The multispectral orhyperspectral medical imaging system of claim 1, wherein each respectivespectral band in the plurality of spectral bands used by the diagnosticprotocol module has a bandwidth of less than 20 nm.
 17. Themultispectral or hyperspectral medical imaging system of claim 1,wherein the particular characteristic of the in vivo tissue enhanced bythe pseudo-color image is cancerous tissue.
 18. The multispectral orhyperspectral medical imaging system of claim 17, wherein the canceroustissue comprises breast cancer or lymphoma.
 19. The multispectral orhyperspectral medical imaging system of claim 1, wherein thepseudo-color image includes information about at least one of thepresence of a tumor, the presence of a residual tumor at a margin of asurgical excision bed, and the progression of a tumor.
 20. Themultispectral or hyperspectral medical imaging system of claim 19,wherein the information about the progression of the tumor includes atleast one of tumor stage grading and microvascular changes in a vasculartissue.